1. Field of the Invention
The invention relates generally to digital image processing and the display of digitally generated images. The invention relates more specifically to the problem of creating raster-based, high-resolution animated images in real time, where the mechanism for generating each raster line is modifiable on a by-the-line or on a by-a-group of lines basis.
2a. Copyright Claims to Disclosed Code
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
In particular, this application includes C language source-code listings of a variety of computer program modules. These modules can be implemented by way of a computer program, microcode, placed in a ROM chip, on a magnetic or optical storage medium, and so forth. The function of these modules can also be implemented at least in part by way of combinatorial logic. Since implementations of the modules which are deemed to be "computer programs" are protectable under copyright law, copyrights not otherwise waived above in said modules are reserved. This reservation includes the right to reproduce the modules in the form of machine-executable computer programs.
2b. Cross Reference to Related Applications
This application is related to:
PCT Patent Application Serial No. PCT/US92/09342, entitled RESOLUTION ENHANCEMENT FOR VIDEO DISPLAY USING MULTI-LINE INTERPOLATION, by inventors Mical et al., filed Nov. 2, 1992, Attorney Docket No. MDIO3050, and also to U.S. patent application Ser. No. 07/970,287, bearing the same title, same inventors and also filed Nov. 2, 1992; PA1 PCT Patent Application Serial No. PCT/US92/09349, entitled AUDIO/VIDEO COMPUTER ARCHITECTURE, by inventors Mical et al., filed Nov. 2, 1992, Attorney Docket No. MDIO4222, and also to U.S. patent application Ser. No. 07/970,308, bearing the same title, same inventors and also filed Nov. 2, 1992; PA1 PCT Patent Application Serial No. PCT/US92/09348, entitled METHOD FOR GENERATING THREE DIMENSIONAL SOUND, by inventor David C. Platt, filed Nov. 2, 1992, Attorney Docket No. MDIO4220, and also to U.S. patent application Ser. No. 07/970,274, bearing the same title, same inventor and also filed Nov. 2, 1992; PA1 PCT Patent Application Serial No. PCT/US92/09350, entitled METHOD FOR CONTROLLING A SPRYTE RENDERING PROCESSOR, by inventors Mical et al., filed Nov. 2, 1992, Attorney Docket No. MDIO3040, and also to U.S. patent application Ser. No. 07/970,278, bearing the same title, same inventors and also filed Nov. 2, 1992; PA1 PCT Patent Application Serial No. PCT/US92/09462, entitled SPRYTE RENDERING SYSTEM WITH IMPROVED CORNER CALCULATING ENGINE AND IMPROVED POLYGON-PAINT ENGINE, by inventors Needle et al., filed Nov. 2, 1992, Attorney Docket No. MDIO4232, and also to U.S. patent application Ser. No. 07/970,289, bearing the same title, same inventors and also filed Nov. 2, 1992; PA1 PCT Patent Application Serial No. PCT/US92/09460, entitled METHOD AND APPARATUS FOR UPDATING A CLUT DURING HORIZONTAL BLANKING, by inventors Mical et al., filed Nov. 2, 1992, Attorney Docket No. MDIO4250, and also to U.S. patent application Ser. No. 07/969,994, bearing the same title, same inventors and also filed Nov. 2, 1992; PA1 PCT Patent Application Serial No. PCT/US92/09467, entitled IMPROVED METHOD AND APPARATUS FOR PROCESSING IMAGE DATA, by inventors Mical et al., filed Nov. 2, 1992, Attorney Docket No. MDIO4230, and also to U.S. patent application Ser. No. 07/970,083, bearing the same title, same inventors and also filed Nov. 2, 1992; and PA1 PCT Patent Application Serial No. PCT/US92/09384, entitled PLAYER BUS APPARATUS AND METHOD, by inventors Needle et al., filed Nov. 2, 1992, Attorney Docket No. MDIO4270, and also to U.S. patent application Ser. No. 07/970,151, bearing the same title, same inventors and also filed Nov. 2, 1992.
The related patent applications are all commonly assigned with the present application and are all incorporated herein by reference in their entirety.
The present application is to be considered a continuation-in-part of one or more of the above cited, co-pending applications, including at least one of: U.S. patent application Ser. No. 07/970,287, filed Nov. 2, 1992 and entitled RESOLUTION ENHANCEMENT FOR VIDEO DISPLAY USING MULTI-LINE INTERPOLATION; U.S. patent application Ser. No. 07/969,994, filed Nov. 2, 1992 and entitled METHOD AND APPARATUS FOR UPDATING A CLUT DURING HORIZONTAL BLANKING; and U.S. patent application Ser. No. 07/970,289, filed Nov. 2, 1992 and entitled SPRYTE RENDERING SYSTEM WITH IMPROVED CORNER CALCULATING ENGINE AND IMPROVED POLYGON-PAINT ENGINE.
3. Description of the Related Art
In recent years, the presentation and prepresentation processing of visual imagery has shifted from what was primarily an analog electronic format to an essentially digital format.
Unique problems come to play in the digital processing of image data and the display of such image data. The more prominent problems include providing adequate storage capacity for digital image data and maintaining acceptable data throughput rates while using hardware of relatively low cost. In addition, there is the problem of creating a sense of realism in digitally generated imagery, particularly in animated imagery.
The visual realism of imagery generated by digital video game systems, simulators and the like can be enhanced by providing special effects such as moving sprites, real-time changes in shadowing and/or highlighting, smoothing of contours and so forth.
Visual realism can be further enhanced by increasing the apparent resolution of a displayed image so that it has a smooth photography-like quality rather than a grainy disjoined-blocks appearance of the type found in low-resolution computer-produced graphics of earlier years.
Visual realism can be even further enhanced by increasing the total number of different colors and/or shades in each displayed frame of an image so that, in regions where colors and/or shades are to change in a smooth continuum by subtle degrees of hue/intensity, the observer perceives such a smooth photography-like variation of hue/intensity rather than a stark and grainy jump from one discrete color/shade to another. Glaring changes of color/shade are part of the reason that computer-produced graphics of earlier years had a jagged appearance rather than a naturally smooth one.
Although bit-mapped computer images originate as a matrix of discrete lit or unlit pixels, the human eye can be fooled into perceiving an image having the desired photography-like continuity if the displayed matrix of independently-shaded (and/or independently colored) pixels has dimensions of approximately 500-by-500 pixels or better at the point of display and a large variety of colors and/or shades on the order of roughly 24 bits-per-pixel or better.
The VGA graphics standard, which is used in many present-day low-cost computer systems, approximates this effect with a display matrix having dimensions of 640-by-480 pixels. However, conventional low-cost VGA graphic systems suffer from a limited per-frame palette of available colors and/or shades.
Standard NTSC broadcast television systems also approximate the continuity mimicking effect by using interlaced fields with 525 lines per pair of fields and a horizontal scan bandwidth (analog) that is equivalent to approximately 500 RGB colored dots per line.
More advanced graphic display standards such as Super-VGA and High Definition Television (HDTV) rely on much higher resolutions, 1024-by-786 pixels for example. It is expected that display standards with even higher resolution numbers (e.g., 2048-by-2048) will emerge in the future. It is expected that the number of bits per displayed pixel will similarly increase in the future.
As resolutions increase, and a wider variety of colors/shades per frame is sought, the problem of providing adequate storage capacity for the corresponding digital image data becomes more acute. The problem of providing sufficient data processing throughput rates also becomes more acute. This is particularly so if an additional constraint is imposed of keeping hardware costs within acceptable price versus performance range.
A display with 640-by-480 independent pixels (307,200 pixels total) calls for a video-speed frame buffer having at least 19 address bits or a corresponding 2.sup.19 independently-addressable data words (=512K words), where each data word stores a binary code representing the shading and/or color of an individual pixel. Each doubling of display resolution, say from 640-by-480 pixels to 1280-by-960 pixels, calls for a four-fold increase in the storage capacity of the frame buffer. Each doubling of per-pixel color/shade variation, say from 8 bits-per-pixel to 16 bits-per-pixel, calls for an additional two-fold increase in storage capacity. This means that a system starting with a display of 8 bits-per-pixel and 640-by-480 independent pixels per screen would conventionally require a memory increase from 512K bytes to 4 MB (four Megabytes) as a result of doubling both the number of pixels per row and column and the number of bits-per-pixel. And in cases where parts or all of the resultant 1280-by-960 display field have to be modified in real-time (to create a sense of animation), the eight-fold increase of storage capacity calls for a corresponding eight-fold increase in data processing bandwidth (image bits processed per second) as compared to what was needed for processing the original, 8 bits-per-pixel, 640-by-480 pixels field.
The benefit versus cost ratio incurred by meeting demands for more storage capacity and faster processing speed has to be questioned at some point. Perhaps a given increase in performance is not worth the increase in system cost. On the other hand, it might be possible to create a perception of improved performance without suffering a concomitant burden of significantly higher cost.
Such an objective can be realized by using a High-performance, Inexpensive, Image-Rendering system (HI-IR system) such as disclosed in the above cited set of co-related patent applications. In particular, part of the low-cost and high-performance of the HI-IR system is owed to the use, in a display-defining path of the system, of a Color LookUp Table (CLUT) whose contents are modifiable on a by-the-line basis. Details of this CLUT system may be found in the above-cited PCT Patent Application Serial No. PCT/US92/09460, entitled METHOD AND APPARATUS FOR UPDATING A CLUT DURING HORIZONTAL BLANKING, by inventors Mical et al., filed Nov. 2, 1992.
Another part of the low-cost and high-performance of the HI-IR system is owed to the use, in the display-defining path of the system, of a subposition-weighted Interpolator whose subposition weights are modifiable on a by-the-pixel basis and whose mode of operation (horizontal-interpolation on/off and vertical-interpolation on/off) is modifiable on a by-the-line or by-the-frame basis.
Yet another part of the low-cost and high-performance of the HI-IR system is owed to the use, in the display-defining path of the system, of a slip-stream mechanism in which "background" pixels can be replaced or not, on a modifiable by-the-line basis, with so-called externally-provided slipstream video data to create a picture-in-picture or another like effect. A description of this slipstream process may be found in the above-cited PCT Patent Application Serial No. PCT/US92/09349, entitled AUDIO/VIDEO COMPUTER ARCHITECTURE, by inventors Mical et al.
Still another part of the low-cost and high-performance of the HI-IR system is owed to the use, in a bitmap-defining portion of the system, of a unique set of one or more "spryte" rendering engines (also called cel animating engines) for executing a list of bitmap modifications stored in a queue. A description of this mechanism may be found in the above cited PCT Patent Application Serial No. PCT/US92/09350, entitled METHOD FOR CONTROLLING A SPRYTE RENDERING PROCESSOR, and also PCT Patent Application Serial No. PCT/US92/09462, entitled SPRYTE RENDERING SYSTEM WITH IMPROVED CORNER CALCULATING ENGINE AND IMPROVED POLYGON-PAINT ENGINE.
The rich assortment of capabilities that are made possible by these and other mechanisms of the HI-IR system provide benefits on the one hand, but create a new set of problems on the other hand.
In particular, it becomes a problem to manage and coordinate attempts by one or more application programs to alter the configuration of the display-defining path of the HI-IR system, or to change the operations of the spryte-rendering portion of the HI-IR system. Each operational change that is made either to the display-defining path of the HI-IR system, or to the spryte-rendering portion of the HI-IR system, can result in desired-beneficial changes to what is shown on the display monitor or it can just as easily produce undesired-detrimental changes to what is shown on the display monitor.
The desired-beneficial changes are, of course, no problem. Examples include the creation of a photography-quality background scene over which animated "spryres" move.
The undesired-detrimental changes can give nontechnical users of the machine a wrong impression of what is happening to their machine. Such users may come to believe that something has become permanently damaged within their machine (even though this is not true) and the users may then come to form a poor opinion of the machine's performance capabilities. It is preferable to give nontechnical users an impression that the machine is "robust" and can perform even under adverse conditions where an ill-behaved application program is installed in the machine.
There are some portions of the display-defining path of the HI-IR system, for example, that should be "configured" one time only, during the power-up/reset phase of machine operation (initialization phase). An example is the setting of a video-display driver within the system to an NTSC television drive mode or a PAL television drive mode. An ill-behaved module within an application program might inadvertently load a new configuration into the system after power-up/reset and thereby cause the entire display to show out-of-synch noise or "garbage". It may not be possible to fix this problem other than by shutting power off and restarting the machine. This type of "fix" is undesirable because it gives nontechnical users a notion that their machine is not as "robust" as they would like it to be. Manufacturers wish to continuously leave consumers with an impression that the machine they purchased is "robust" and is able to continue functioning in some minimal way even if loaded with an ill-behaved application programs.
On the other hand, manufacturers wish to make machines that are easily reconfigured to meet the requirements of specific markets. Systems sold in the United States are preferably configured, for example, to conform to the NTSC television standard while systems sold in Europe are preferably configured to conform to the PAL television standard.
A first presented problem is therefore how to permit easy reconfiguration of machines to conform with standards of different markets and yet at the same time avoid the appearance of less than robust, machine performance even in the case where an ill-behaved application program manages to enter the system.
Another problem relates to making sure that certain post-initialization reconfigurations of the display-defining path of the HI-IR system are carried in a timely manner and coordinated with operations of the spryte rendering engines. Some operations of the display-defining path of the HI-IR system and of the spryte rendering engines are preferably modified or "reconfigured" on a by-the-frame basis, or on a by-the-line basis. These modifications/reconfigurations should be coordinated with real-time events of the display-defining path of the system such as the actuation of the horizontal synch and vertical synch pulses of the video generating system.
In some situations, it is undesirable to let reconfiguration of a displayed image occur in the middle of an active scan line. This might create a clearly visible and annoying "tear" artifact or other disturbance in the displayed imagery. Ideally, reconfiguration should occur during the vertical blanking or horizontal blanking periods of the system so as to avoid the image-tearing problem.
On the other hand, the performance speed of real-time games or simulations might suffer if one always had to wait for the next horizontal or vertical blanking period each time a change was to be made. Some kinds of imagery changes can be made without creating a noticeable disturbance within the displayed image while others cannot. A flexible mechanism is needed for allowing both kinds of changes.
Another problem presented here is therefore, how to efficiently organize and prioritize the execution of real-time image and modality changes on a by-the-line or by-the-frame basis. A method is needed for coordinating and prioritizing changes to be made to the display-defining path of the HI-IR system and changes made by the spryte-rendering portion of the system.